Location recognition method and apparatus in atypical environment

ABSTRACT

Provided is an atypical environment-based location recognition apparatus. The apparatus includes a sensing information acquisition unit configured to, from sensing data collected by sensor modules, detect object location information and semantic label information of a video image and detect an event in the video image; a walk navigation information provision unit configured to acquire user movement information; a metric map generation module configured to generate a video odometric map using sensing data collected through a sensing information acquisition unit and reflect the semantic label information; and a topology map generation module configured to generate a topology node using sensing data acquired through the sensing information acquisition unit and update the topology node through the collected user movement information.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to and the benefit of Korean PatentApplication No. 2020-0176412, filed on Dec. 16, 2020, the disclosure ofwhich is incorporated herein by reference in its entirety.

BACKGROUND 1. Field of the Invention

The present invention relates to an atypical environment-based locationrecognition apparatus, and more specifically, an atypicalenvironment-based location recognition apparatus capable of graduallygenerating a map of a corresponding area, and at the same time,estimating precise locations in a moving object that moves in aGNSS-denied environment or in a building or underground facility that itenters for the first time without prior information.

2. Discussion of Related Art

As frontline operations have recently changed to focus on urbanoperations such as small-scale and counter-terrorism responses, it isessential to support advanced information communications technologies(ICT) that can be utilized when soldiers are engaged in their usualoperational training and actual combat.

In particular, location information of soldiers is information that isessential to secure in training or actual combat, and many technicalapproaches have been attempted for the location information.

In most outdoor areas, a satellite navigation system that acquireslocation information using GNSS satellite signals is used, but thiscannot be used in dense building areas or indoor spaces. Thus, a methodof using wireless signals, a method of using a sensor, and the like havebeen proposed.

For general pedestrians, walking navigation using wireless signalanalysis based on WiFi, BLE, mobile communication networks, and the likeand sensor signal analysis based on an accelerometer and a gyro sensoris generally applied. However, a method of estimating the location of aterminal using IR-UWB, a method of merging walking navigation, and thelike have been proposed for environments where infrastructure cannot beutilized.

With the technological advances of artificial intelligence and videorecognition in recent years, interest in technology for performingsimultaneous localization and mapping (SLAM) based on vision sensors isincreasing.

SLAM is a technology that uses information acquired from sensors tocreate a map of the surrounding environment of a moving object, and atthe same time, estimate the current location of the moving object.Various sensors (lidar, radar, laser, camera, etc.) may be used.

SLAM is a technology for estimating a user's location while creating amap of the environment when driving in an unknown space without priorinformation. Various methodologies have been studied. Thesemethodologies had a common problem that real-time performance is low dueto a large amount of computation. However, by integrating deep learning,which is the core of recent technological progress in the computervision field, opportunities to increase the performance of the SLAMtechnology have become available.

Video SLAM (hereafter referred to as VSLAM) is a real-time version ofStructure from Motion (SfM), which solves the SLAM problem using avisual odometric map.

Oriented FAST and Rotated BRIEF (ORB-SLAM), which is one of thehandcraft feature point-based methods, is a method capable of utilizingan ORB feature point to compute camera trajectories and a 3D map invarious environments.

Also, LSD-SLAM, which is one of the direct video-based methods, is amethod of performing estimation using all image information withoutextracting key points or feature points.

This method provides higher accuracy and robustness in environmentswhere it is difficult to extract key points or features, such asindoors, and enables a relatively denser 3D reconstruction.

In addition, conventional video-based driving path distance estimationmethods rely on a handcraft feature point-based method, and thesemethods are optimized for the motion of objects with a large movingdistance or stable acceleration or deceleration during the movement,such as automobiles, flying objects, or robots.

In addition, the conventional methods have a problem in that they arevery vulnerable to real-time changing environments, motions of nearbyobjects, and relocation of objects,

Meanwhile, a pedestrian is in an environment in which nearby people orobjects change every moment, and a combatant's movement pattern itselfis in a very dynamic environment in which movement distances are small,but changes in acceleration or deceleration in actual behavior are verylarge.

Accordingly, conventional video-based driving path distance estimationscan hardly be applied to general pedestrian applications in whichdynamic motions are random.

SUMMARY OF THE INVENTION

The present invention is to solve the conventional problems and isdirected to providing an atypical environment-based location recognitionapparatus capable of gradually generating a 3D map of an operationalarea, and at the same time, providing precise locations through thecooperation of combatants in a building or underground facility that isentered for the first time without prior information, in a GNSS-deniedenvironment, in poor quality data collection due to irregular anddynamic motions of combatants, and in modified battlefield spaceconditions.

The present invention is also directed to providing an atypicalenvironment-based location recognition apparatus capable of reducing thetime required to build a map through a metric map using robust locationrecognition and semantic information that merge a multi-sensor-basedVSLAM function and a walking navigation function and also reducingpositional errors in order to maintain a certain level of performanceover time by configuring a map to be robust against environmentalchanges.

The present invention is not limited to the above objectives, and otherobjectives not described herein may be clearly understood by thoseskilled in the art from the following description.

According to an aspect, there is provided an atypical environment-basedlocation recognition apparatus including a sensing informationacquisition unit configured to collect sensing data including a videoimage from sensor modules, a walking navigation information provisionunit configured to acquire user movement information, a video analysisunit configured to detect object location information and semantic labelinformation from the video image and analyze whether an event isdetected in the video image, a metric map generation module configuredto generate a video odometric map using sensing data collected throughthe sensing information acquisition unit and information analyzedthrough the video analysis unit and then reflect the semantic labelinformation; and a topology map generation module configured to generatea topology node using sensing data acquired through the sensinginformation acquisition unit and update the topology node throughcollected user movement information.

The topology map generation module may include a node generation unitconfigured to generate a topology node through the semantic labelinformation and object location information acquired through the sensinginformation acquisition unit, a transition determination unit configuredto analyze location received event information and an actual user'slocation information to determine whether there is a need for nodetransition, a node management unit configured to update a location ofthe generated topology node according to whether to perform thetransition, and a map merge unit configured to compare semantic labelinformation provided through the metric map generation module to thetopology node to merge the topology node and user location information.

Also, the topology map generation module may further include aself-supervised learning unit configured to perform self-supervisedlearning for a series of metric map functions generated from the user'strajectory.

The topology map generation module may further include a reinforcementlearning unit configured to perform reinforcement learning when a metricmap function sequence is generated in a path in which the topology nodeis generated.

The walking navigation information provision unit may acquire usermovement information including the user's stride length, movementdirection, and movement distance data and provide the user movementinformation to the topology map generation module.

The transition determination unit may include a node determination unitconfigured to determine whether the semantic label information of themetric map acquired through the video image corresponds to nodeinformation of a topology map on the basis of the user's locationacquired through the sensing information acquisition unit, a nodecorrection unit configured to correct the node information of thetopology map when a determination result of the node determination unitis that the semantic label information of the metric map does notcorrespond to a node of the topology map, a link processing unitconfigured to add a link between the node of the topology map and asubsequent node when a determination result of the node determinationunit is that the semantic label information of the metric mapcorresponds to the node of the topology map, a link distance computationunit configured to compute a distance of the added link through the usermovement information provided from the walking navigation module, and anode transition unit configured to correct a location of the node of thetopology map and the added link to correspond to the link distancecomputed by the link distance computation unit.

The topological map generation module may use uncorrected topology mapinformation generated using actual map information.

According to another aspect, there is provided an atypicalenvironment-based location recognition method including a sensinginformation acquisition operation for, from sensing data collected bysensor modules detecting object location information and semantic labelinformation of a video image and detecting an event in the video image,acquiring user movement information, generating a video odometric mapusing the collected sensing data through a metric map generation moduleand reflecting the semantic label information, and generating a topologynode using the collected sensing data through a topology map generationmodule and updating the topology node through the user movementinformation and the video odometric map.

The updating of the topology node may include generating, by a nodegeneration unit, the topology node through the acquired semantic labelinformation and object location information; analyzing, by a transitiondetermination unit, received event information and an actual user'slocation information to determine whether there is a need for nodetransition; updating, by a node management unit, a location of thegenerated topology node according to whether to perform the transition;and comparing, by a map merge unit, the semantic label informationprovided through the metric map generation module to the topology nodeto merge the topology node and user location information.

The atypical environment-based location recognition method may furtherinclude performing, by a self-supervised learning unit, self-supervisedlearning for a series of metric map functions generated from the user'strajectory.

The atypical environment-based location recognition method may furtherinclude performing, by a reinforcement learning unit, reinforcementlearning when a metric map function sequence is generated in a path inwhich the topology node is generated.

The acquiring of user movement information may include acquiring usermovement information including the user's stride length, movementdirection, and movement distance data and providing the user movementinformation to a topology map generation module.

The determining of whether to transition between a node and a userlocation may include determining, by a node determination unit, whetherthe semantic label information of the metric map acquired through thevideo image corresponds to node information of a topology map on thebasis of the user's location acquired through a sensing informationacquisition unit; correcting, by a node correction unit, the nodeinformation of the topology map when the semantic label information ofthe metric map does not correspond to a node of the topology map in thedetermination operation; adding, by a link processing unit, a linkbetween the node of the topology map and a subsequent node when thesemantic label information of the metric map corresponds to the node ofthe topology map in the determination operation; computing, by a linkdistance computation unit, a distance of the added link through the usermovement information; and correcting, by a node transition unit, alocation of the node of the topology map and the added link tocorrespond to the computed link distance.

The generating of a topology node may include using uncorrected topologymap information generated using actual map information.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a functional block diagram illustrating an atypicalenvironment-based location recognition apparatus according to thepresent invention.

FIG. 2 is a reference diagram illustrating a map building time using asimultaneous localization and mapping (SLAM) function.

FIG. 3 is a block diagram illustrating a topology map generation moduleaccording to an embodiment of the present invention.

FIG. 4 is a block diagram illustrating a transition determination unitof FIG. 3.

FIG. 5 is a reference diagram illustrating a process of generating atime-wise topology map corresponding to a user's movement by an atypicalenvironment-based location recognition apparatus according to anembodiment of the present invention.

FIG. 6 is a reference diagram illustrating uncorrected topology mapinformation that is basically generated before user movement informationis acquired according to another embodiment of the present invention.

FIG. 7 is a reference diagram illustrating a map of an actual building.

FIG. 8 is a reference diagram illustrating a topology map with nodes andlinks corrected by a walking navigation information provision unit ofFIG. 1.

FIG. 9 is a flowchart illustrating an atypical environment-basedlocation recognition apparatus according to an embodiment of the presentinvention.

FIG. 10 is a flowchart illustrating an operation of updating a topologynode of FIG. 9.

FIG. 11 is a flowchart illustrating an operation of determining whetherthere is a transition between a node and a user location of FIG. 10.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

Advantages and features of the present invention, and implementationmethods thereof will be clarified through the following embodimentsdescribed in detail with reference to the accompanying drawings.However, the present invention is not limited to embodiments disclosedherein and may be implemented in various different forms. Theembodiments are provided for making the disclosure of the preventioninvention thorough and for fully conveying the scope of the presentinvention to those skilled in the art. It is to be noted that the scopeof the present invention is defined by the claims. The terminology usedherein is for the purpose of describing particular embodiments only andis not intended to be limiting of the invention. Herein, the singularshall be construed to include the plural, unless the context clearlyindicates otherwise. The terms “comprises” and/or “comprising” as usedherein specify the presence of stated elements, steps, operations,and/or components but do not preclude the presence or addition of one ormore other elements, steps, operations, and/or components.

FIG. 1 is a functional block diagram illustrating an atypicalenvironment-based location recognition apparatus according to thepresent invention. As shown in FIG. 1, the atypical environment-basedlocation recognition apparatus according to an embodiment of the presentinvention may include a sensing information acquisition unit 100, awalking navigation information provision unit 200, a video analysis unit300, a topology map generation module 400, and a metric map generationmodule 500.

The sensing information acquisition unit 100 collects sensing dataincluding a video image from sensor modules. The sensing informationacquisition unit 100 may include a radar, a lidar, a laser, an infrareddevice, a stereo camera, an RGB-D camera, and inertial measurement unit(IMU) sensors and may use the collected sensing data. Here, an event inan image means whether a user is moving.

The walking navigation information provision unit 200 acquires usermovement information from the sensing data collected through the sensinginformation acquisition unit 100. Here, the user movement informationmeans a user's stride length, movement direction, and movement distance.In this embodiment, the user movement information may be calculated fromsensing data acquired through an IMU sensor.

The video analysis unit 300 detects semantic label information andobject position information from a sensed video image and analyzeswhether an event is detected in the video image.

The metric map generation module 500 generates a video odometric mapusing the sensing data collected through the sensing informationacquisition unit 100 and the information analyzed through the videoanalysis unit 300 and then reflects semantic label information. That is,the metric map generation module 500 reflects the semantic labelinformation in a node detected through the generated video odometricmap. Here, the node indicates an independent space surrounded by a wall,such as a room.

The metric map generation module 500, which uses a visual simultaneouslocalization and mapping (VSLAM) function, typically uses low-levelimage features (i.e., corners, blobs, etc.) to estimate a user'slocation and generate a metric map.

As shown by the blue line in FIG. 2, it is possible to distinguish aninitial map building operation in which map quality is improved and asubsequent development operation in which the quality is maintained. Inaddition, as shown by the red line, in a harsh environment such as abattlefield environment, it takes much more time to build a usable-levelmap, and it is also difficult to maintain the usable level. Inparticular, changes in illuminance, blurred videos, etc. that may occurin a situation in which dynamic motion is intense increase the timerequired to build a metric map.

Also, the metric map depends on primary image characteristics. Thus, themetric map is very sensitive to environmental changes and requires morefrequent updates in places where the environment changes frequently. Thegreen line, which indicates the most ideal case, shows a situation inwhich the map is quickly and safely built.

The topology map generation module 400 generates a topology node usingsensing data acquired through the sensing information acquisition unit100 and updates the topology node through collected user movementinformation. In an embodiment of the present invention, the topology mapis a simplified representation of the space, and the target space isexpressed at a more condensed level.

FIG. 3 is a block diagram illustrating a topology map generation module400 according to an embodiment of the present invention.

As shown in FIG. 3, the topology map generation module 400 according toan embodiment of the present invention includes a node generation unit410, a transition determination unit 420, a node management unit 430,and a map merge unit 440.

The node generation unit 410 generates a topology node through objectlocation information and semantic label information acquired through thevideo analysis unit 300.

The transition determination unit 420 analyzes received eventinformation and an actual user's location information to determinewhether there is a need for node transition.

The node management unit 430 updates the location of the generatedtopology node according to whether to perform transition.

The map merge unit 440 compares the topology node and the semantic labelinformation provided through the metric map generation module 500,merges the topology node and the user location information, and providesthe merged topology node and user location information.

FIG. 4 is a block diagram illustrating the transition determination unitof FIG. 3.

As shown in FIG. 4, the transition determination unit 420 includes anode determination unit 421, a node correction unit 422, a linkprocessing unit 423, a link distance computation unit 424, and a nodetransition unit 425.

The node determination unit 421 determines whether the semantic labelinformation of the metric map acquired through the video imagecorresponds to node information of the topology map on the basis of auser's location acquired through the sensing information acquisitionunit 100.

When a result of the determination of the node determination unit 421 isthat the semantic label information of the metric map does notcorrespond to the node of the topology map, the node correction unit 422corrects subsequent node information of the topology map at the currentuser location.

As an example, when there is no node of the topology map but relevantinformation is present in the semantic label information of the metricmap, the node correction unit 422 additionally constructs a node of thetopology map. When there is a node of the topology map but relevantinformation is not present in the semantic label information of themetric map, the node correction unit 422 performs correction to deletethe node of the topology map.

Meanwhile, when a determination result of the determination unit is thatthe semantic label information of the metric map corresponds to the nodeof the topology map, the link processing unit 423 adds a link betweenthe node of the topology map and a subsequent node.

The link distance computation unit 424 computes a distance of the addedlink through user movement information provided from the walkingnavigation module.

The node transition unit 425 corrects the node location of the topologymap and the distance of the added link to correspond to the linkdistance computed by the link distance computation unit 424.

As described above, since the topology map is in a state in whichgeometric relationships are excluded, user movement informationgenerated by the walking navigation module, that is, values such asstride length, movement direction, and movement distance, is received tocorrect the link length between nodes and adjust the relative positionsof the nodes.

FIG. 5 is a reference diagram illustrating a process of generating atime-wise topology map corresponding to a user's movement by an atypicalenvironment-based location recognition apparatus according to anembodiment of the present invention.

As shown in FIG. 5, according to an embodiment of the present invention,an atypical environment-based location recognition apparatus generates atopology map while moving over time. For example, as shown in FIG. 5, acircle, which is a node, may represent an independent space surroundedby a wall, such as a room, and a line may represent a link which is apassage connecting the rooms to each other.

Therefore, when training is performed to pre-detect video data fromsensing data for various types of buildings and generate a topology mapon the basis of the detected video data, the topology map creates atop-level graph more quickly than a metric map. Thus, it is possible tocreate and expand a node and a link according to the characteristics ofa space in which soldiers are moving.

First, when a location recognition apparatus enters a building (t1), thetopology map generation module 400 generates a virtual node of thetopology map.

Subsequently, while the location recognition apparatus moves (t2) afterentering the building, the metric map generation module 500 detects anode (an independent space surrounded by a wall, such as a room) throughVSLAM and reflects semantic label information in the detected node.

When a node is detected through VSLAM, the topology map generationmodule 400 compares and matches the generated semantic label informationof the node to the node detected through the metric map generationmodule 500.

Subsequently, the location recognition apparatus moves (t3), and themetric map generation module 500 consequently detects a subsequent nodethrough VSLAM.

When no subsequent node is present in the generated topology map as in“t3,” the topology map generation module 400 newly generates a virtualnode.

In this case, the topology map generation module 400 corrects thelocation of the subsequent node of the topology map as in “t4” usinguser movement information acquired through the walking navigationinformation provision unit 200.

According to such an embodiment of the present invention, by a locationrecognition apparatus generating a topology map while moving into abuilding, when location recognition is to be performed in an atypicalenvironment with dynamically high acceleration/deceleration, it ispossible to generate an environment map more quickly and accurately evenat a place visited for the first time and recognize a user's location inthe place.

Also, according to an embodiment of the present invention, the minimumlocation recognition information may be provided in an environment whereinfrastructure has been lost or where wireless propagation is not good,and thus it is possible to make a significant contribution to improvingthe survivability of soldiers performing operations in the future.

Meanwhile, as shown in FIG. 6, a topology map generation moduleaccording to another embodiment of the present invention may receiveuncorrected topology map information (nodes 1 to 6) using mapinformation and may store and manage the uncorrected topology mapinformation in a separate memory.

Based on the uncorrected topology map provided in this way, usermovement information and semantic label information for a node of avideo odometric map to be detected by searching an actual building ofFIG. 7 which is detected by the metric map generation module 500 may beused to generate a topology map in which the locations of the nodes(nodes 1 to 6) are corrected as shown in FIG. 8 and to correct thelength and location of a link connecting the nodes.

Meanwhile, the topology map generation module 400 may performself-supervised learning for a series of metric map functions generatedfrom the user's trajectory or may perform reinforcement learning whengenerating a metric map function sequence in a path in which thetopology node is generated.

An atypical environment-based location recognition method according toan embodiment of the present invention will be described below withreference to FIG. 9.

First, from sensing data collected by sensor modules, the video analysisunit 300 detects object location information and semantic labelinformation of a video image and detects an event in the video image(S110).

Subsequently, a walking navigation information provision unit 200acquires user movement information (S120). In the operation of acquiringthe user movement information (S120), user movement informationincluding the user's stride length, movement direction, and movementdistance data is acquired and provided to a topology map generationmodule.

Subsequently, the metric map generation module 500 generates a videoodometric map using the collected sensing data and then reflects thesemantic label information (S130).

Subsequently, the topology map generation module 400 generates atopology node using the acquired sensing node and updates the topologynode through collected user movement information and the video odometricmap (S140).

Sub-operations of the operation of updating the topology node (S140)according to an embodiment of the present invention will be describedbelow with reference to FIG. 10.

The node generation unit generates a topology node through the acquiredsemantic label information and object location information (S141).

Subsequently, the transition determination unit analyzes received eventinformation and an actual user's location information to determinewhether there is a need for node transition (S142).

Also, the node management unit updates the location of the generatedtopology node according to whether to perform transition (S143).

Subsequently, the map merge unit compares the topology node and thesemantic label information provided through the metric map generationmodule 500 and merges the topology node and the user locationinformation (S144).

Sub-operations of the operation of determining whether there is a needfor transition between a node and a user's location (S142) according toan embodiment of the present invention will be described below withreference to FIG. 11.

The node determination unit determines whether the semantic labelinformation of the metric map acquired through the video imagecorresponds to node information of the topology map on the basis of auser's location acquired through the sensing information acquisitionunit 100 (S1421).

When the semantic label information of the metric map does notcorrespond to the node of the topology map in the determinationoperation S1421 (NO), the node correction unit corrects the nodeinformation the topology map (S1422).

When the semantic label information of the metric map corresponds to thenode of the topology map in the determination operation S1421 (YES), thelink processing unit adds a link between the node of the topology mapand a subsequent node (S1423).

Subsequently, the link distance computation unit computes a distance ofthe added link through provided user movement information (S1424).

The node transition unit corrects a node location of the topology mapand the added link to correspond to the computed link distance (S1425).

According to an embodiment of the present invention, when locationrecognition is to be performed in an atypical environment withdynamically high acceleration/deceleration, it is possible to quicklyand accurately generate an environment map even in a place visited forthe first time.

Also, according to an embodiment of the present invention, the minimumlocation recognition information may be provided in an environment whereinfrastructure has been lost or where wireless propagation is not good,and thus it is possible to make a significant contribution to improvingthe survivability of soldiers performing operations in the future.

A self-supervised learning unit may perform self-supervised learning fora series of metric map functions generated from the user's trajectory ormay perform reinforcement learning when generating a metric map functionsequence in a path in which the topology node is generated.

Meanwhile, as shown in FIG. 6, a topology map generation moduleaccording to another embodiment of the present invention may receiveuncorrected topology map information (nodes 1 to 6) using mapinformation and may store and manage the uncorrected topology mapinformation in a separate memory.

Based on the uncorrected topology map provided in this way, usermovement information and semantic label information for a node of avideo odometric map obtained by sensing an actual building (FIG. 7)which is detected by the metric map generation module 500 may be used togenerate a topology map in which the locations of the nodes (nodes 1 to6) are corrected as shown in FIG. 8 and to correct the length andlocation of a link connecting the nodes.

For reference, the elements according to an embodiment of the presentinvention may be implemented as software or hardware such as a fieldprogrammable gate array (FPGA) or an application specific integratedcircuit (ASIC) and may perform predetermined roles.

However, the elements are not limited to software or hardware and may beconfigured to be in an addressable storage medium or configured toactivate one or more processors.

Accordingly, as an example, the elements include elements such assoftware elements, object-oriented software elements, class elements,and task elements, processes, functions, attributes, procedures,subroutines, program code segments, drivers, firmware, microcode,circuits, data, databases, data structures, tables, arrays, andvariables.

Elements and functions provided by corresponding elements may becombined into a smaller number of elements or may be divided intoadditional elements.

It will be understood that each block of the flowcharts and/or blockdiagrams, and combinations of blocks in the flowcharts and/or blockdiagrams, can be implemented by computer program instructions. Thesecomputer program instructions may be provided to a processor of ageneral-purpose computer, special purpose computer, or otherprogrammable data processing apparatus, such that the instructions,which are executed via the processor of the computer or otherprogrammable data processing apparatus, create means for implementingthe functions specified in a flowchart block(s). These computer programinstructions may also be stored in a computer-accessible orcomputer-readable memory that can direct a computer or otherprogrammable data processing apparatus to function in a particularmanner, such that the instructions stored in the computer-accessible orcomputer-readable memory can also produce articles of manufactureembedding instruction means which implement the functions specified inthe flowchart block(s). The computer program instructions may also beloaded onto a computer or other programmable data processing apparatusto cause a series of operations to be performed on the computer or otherprogrammable data processing apparatus to produce a computer-implementedprocess, such that the instructions, which are executed on the computerand other programmable data processing apparatus, can also provideoperations for implementing the functions specified in the flowchartblock(s).

Also, each block described herein may indicate a portion of a module,segment, or code including one or more executable instructions toexecute a specific logical function(s). Moreover, it should be notedthat the functions of the blocks may be performed in a different orderin several modifications. For example, two successive blocks may beperformed substantially at the same time, or they may be performed inreverse order according to their functions.

The term “unit” used herein refers to a software element or a hardwareelement such as an FPGA or an ASIC, and the “unit” performs any role.However, the term “unit” is not limited to software or hardware. A“unit” may be configured to be in an addressable storage medium or toexecute one or more processors. Therefore, for example, the “unit”includes elements, such as software elements, object-oriented elements,class elements, and task elements, processes, functions, attributes,procedures, sub routines, segments of a program code, drivers, firmware,a microcode, a circuit, data, a database (DB), data structures, tables,arrays, and parameters. Furthermore, functions provided in elements and“units” may be combined as a smaller number of elements and “units” orfurther divided into additional elements and “units.” In addition,elements and “units” may be implemented to execute one or more centralprocessing units (CPUs) in a device or secure multimedia card.

According to such an embodiment of the present invention, by a locationrecognition apparatus generating a topology map while moving into abuilding, when location recognition is to be performed in an atypicalenvironment with dynamically high acceleration/deceleration, it ispossible to generate an environment map more quickly and accurately evenat a place visited for the first time and recognize a user's location inthe place.

The configuration of the present invention has been described above indetail with reference to the accompanying drawings, but this is merelyan example. It will be appreciated that those skilled in the art canmake various modifications and changes within the scope of the technicalspirit of the present invention. Therefore, the scope of the presentinvention should not be limited to the above-described embodiments andshould be defined by the appended claims.

What is claimed is:
 1. An atypical environment-based locationrecognition apparatus comprising: a sensing information acquisition unitconfigured to collect sensing data including a video image from sensormodules; a walking navigation information provision unit configured toacquire user movement information; a video analysis unit configured todetect object location information and semantic label information fromthe video image and analyze whether an event is detected in the videoimage; a metric map generation module configured to generate a videoodometric map using sensing data collected through the sensinginformation acquisition unit and information analyzed through the videoanalysis unit and then reflect the semantic label information; and atopology map generation module configured to generate a topology nodeusing sensing data acquired through the sensing information acquisitionunit and update the topology node through collected user movementinformation.
 2. The atypical environment-based location recognitionapparatus of claim 1, wherein the topology map generation modulecomprises: a node generation unit configured to generate the topologynode through the semantic label information and object locationinformation acquired through the sensing information acquisition unit; atransition determination unit configured to analyze received eventinformation and an actual user's location information to determinewhether there is a need for node transition; a node management unitconfigured to update a location of the generated topology node accordingto whether to perform the transition; and a map merge unit configured tocompare semantic label information provided through the metric mapgeneration module to the topology node to merge the topology node anduser location information.
 3. The atypical environment-based locationrecognition apparatus of claim 2, wherein the topology map generationmodule further comprises a self-supervised learning unit configured toperform self-supervised learning for a series of metric map functionsgenerated from the user's trajectory.
 4. The atypical environment-basedlocation recognition apparatus of claim 2, wherein the topology mapgeneration module further comprises a reinforcement learning unitconfigured to perform reinforcement learning when a metric map functionsequence is generated in a path in which the topology node is generated.5. The atypical environment-based location recognition apparatus ofclaim 1, wherein the walking navigation information provision unitacquires user movement information including the user's stride length,movement direction, and movement distance data and provides the usermovement information to the topology map generation module.
 6. Theatypical environment-based location recognition apparatus of claim 2,wherein the transition determination unit comprises: a nodedetermination unit configured to determine whether the semantic labelinformation of the metric map acquired through the video imagecorresponds to node information of a topology map on the basis of theuser's location acquired through the sensing information acquisitionunit; a node correction unit configured to correct the node informationof the topology map when a determination result of the nodedetermination unit is that the semantic label information of the metricmap does not correspond to a node of the topology map; a link processingunit configured to add a link between the node of the topology map and asubsequent node when a determination result of the node determinationunit is that the semantic label information of the metric mapcorresponds to the node of the topology map; a link distance computationunit configured to compute a distance of the added link through the usermovement information provided from the walking navigation module; and anode transition unit configured to correct a location of the node of thetopology map and the added link to correspond to the link distancecomputed by the link distance computation unit.
 7. The atypicalenvironment-based location recognition apparatus of claim 1, wherein thetopological map generation module uses uncorrected topology mapinformation generated using actual map information.
 8. An atypicalenvironment-based location recognition method comprising: a sensinginformation acquisition operation for, from sensing data collected bysensor modules, detecting object location information and semantic labelinformation of a video image and detecting an event in the video image;acquiring user movement information; generating a video odometric mapusing the collected sensing data through a metric map generation moduleand reflecting the semantic label information; and generating a topologynode using the collected sensing data through a topology map generationmodule and updating the topology node through the user movementinformation and the video odometric map.
 9. The atypicalenvironment-based location recognition method of claim 8, wherein theupdating of the topology node comprises: generating, by a nodegeneration unit, the topology node through the acquired semantic labelinformation and object location information; analyzing, by a transitiondetermination unit, received event information and an actual user'slocation information to determine whether there is a need for nodetransition; updating, by a node management unit, a location of thegenerated topology node according to whether to perform the transition;and comparing, by a map merge unit, the semantic label informationprovided through the metric map generation module to the topology nodeto merge the topology node and user location information.
 10. Theatypical environment-based location recognition method of claim 9,further comprising performing, by a self-supervised learning unit,self-supervised learning for a series of metric map functions generatedfrom the user's trajectory.
 11. The atypical environment-based locationrecognition method of claim 9, further comprising performing, by areinforcement learning unit, reinforcement learning when a metric mapfunction sequence is generated in a path in which the topology node isgenerated.
 12. The atypical environment-based location recognitionmethod of claim 8, wherein the acquiring of user movement informationcomprises acquiring user movement information including the user'sstride length, movement direction, and movement distance data andproviding the user movement information to a topology map generationmodule.
 13. The atypical environment-based location recognition methodof claim 9, wherein the determining of whether to transition between anode and a user location comprises: determining, by a node determinationunit, whether the semantic label information of the metric map acquiredthrough the video image corresponds to node information of a topologymap on the basis of the user's location acquired through a sensinginformation acquisition unit; correcting, by a node correction unit, thenode information of the topology map when the semantic label informationof the metric map does not correspond to a node of the topology map inthe determination operation; adding, by a link processing unit, a linkbetween the node of the topology map and a subsequent node when thesemantic label information of the metric map corresponds to the node ofthe topology map in the determination operation; computing, by a linkdistance computation unit, a distance of the added link through the usermovement information; and correcting, by a node transition unit, alocation of the node of the topology map and the added link tocorrespond to the computed link distance.
 14. The atypicalenvironment-based location recognition method of claim 8, wherein thegenerating of a topology node comprises using uncorrected topology mapinformation generated using actual map information.